<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <title>Test::Harness::Beyond - Beyond make test - Perldoc Browser</title>
    <link rel="search" href="https://perldoc.perl.org/opensearch.xml" type="application/opensearchdescription+xml" title="Perldoc Browser">
    <link rel="canonical" href="https://perldoc.perl.org/Test::Harness::Beyond">
    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css" integrity="sha384-JcKb8q3iqJ61gNV9KGb8thSsNjpSL0n8PARn9HuZOnIxN0hoP+VmmDGMN5t9UJ0Z" crossorigin="anonymous">
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/10.5.0/styles/stackoverflow-light.min.css" integrity="sha512-cG1IdFxqipi3gqLmksLtuk13C+hBa57a6zpWxMeoY3Q9O6ooFxq50DayCdm0QrDgZjMUn23z/0PMZlgft7Yp5Q==" crossorigin="anonymous" />
    <style>
      body {
        background: #f4f4f5;
        color: #020202;
      }
      .navbar-dark {
        background-image: -webkit-linear-gradient(top, #005f85 0, #002e49 100%);
        background-image: -o-linear-gradient(top, #005f85 0, #002e49 100%);
        background-image: linear-gradient(to bottom, #005f85 0, #002e49 100%);
        filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff005f85', endColorstr='#ff002e49', GradientType=0);
        background-repeat: repeat-x;
      }
      .navbar-dark .navbar-nav .nav-link,
      .navbar-dark .navbar-nav .nav-link:focus { color: #fff }
      .navbar-dark .navbar-nav .nav-link:hover { color: #ffef68 }
      #wrapperlicious {
        margin: 0 auto;
        font: 0.9em 'Helvetica Neue', Helvetica, sans-serif;
        font-weight: normal;
        line-height: 1.5em;
        margin: 0;
        padding: 0;
      }
      #wrapperlicious h1 { font-size: 1.5em }
      #wrapperlicious h2 { font-size: 1.3em }
      #wrapperlicious h3 { font-size: 1.1em }
      #wrapperlicious h4 { font-size: 0.9em }
      #wrapperlicious h1, #wrapperlicious h2, #wrapperlicious h3,
      #wrapperlicious h4, #wrapperlicious dt {
        color: #020202;
        margin-top: 1em;
        margin-bottom: 1em;
        position: relative;
        font-weight: bold;
      }
      #wrapperlicious a { color: inherit; text-decoration: underline }
      #wrapperlicious #toc { text-decoration: none }
      #wrapperlicious a:hover { color: #2a2a2a }
      #wrapperlicious a img { border: 0 }
      #wrapperlicious :not(pre) > code {
        color: inherit;
        background-color: rgba(0, 0, 0, 0.04);
        border-radius: 3px;
        font: 0.9em Consolas, Menlo, Monaco, monospace;
        padding: 0.3em;
      }
      #wrapperlicious dd {
        margin: 0;
        margin-left: 2em;
      }
      #wrapperlicious dt {
        color: #2a2a2a;
        font-weight: bold;
        margin-left: 0.9em;
      }
      #wrapperlicious p {
        margin-bottom: 1em;
        margin-top: 1em;
      }
      #wrapperlicious li > p {
        margin-bottom: 0;
        margin-top: 0;
      }
      #wrapperlicious pre {
        border: 1px solid #c1c1c1;
        border-radius: 3px;
        font: 100% Consolas, Menlo, Monaco, monospace;
        margin-bottom: 1em;
        margin-top: 1em;
      }
      #wrapperlicious pre > code {
        display: block;
        background-color: #f6f6f6;
        font: 0.9em Consolas, Menlo, Monaco, monospace;
        line-height: 1.5em;
        text-align: left;
        white-space: pre;
        padding: 1em;
      }
      #wrapperlicious dl, #wrapperlicious ol, #wrapperlicious ul {
        margin-bottom: 1em;
        margin-top: 1em;
      }
      #wrapperlicious ul {
        list-style-type: square;
      }
      #wrapperlicious ul ul {
        margin-bottom: 0px;
        margin-top: 0px;
      }
      #footer {
        font-size: 0.8em;
        padding-top: 0.5em;
        text-align: center;
      }
      #more {
        display: inline;
        font-size: 0.8em;
      }
      #perldocdiv {
        background-color: #fff;
        border: 1px solid #c1c1c1;
        border-bottom-left-radius: 5px;
        border-bottom-right-radius: 5px;
        margin-left: auto;
        margin-right: auto;
        padding: 3em;
        padding-top: 1em;
        max-width: 960px;
      }
      #moduleversion { float: right }
      #wrapperlicious .leading-notice {
        font-style: italic;
        padding-left: 1em;
        margin-top: 1em;
        margin-bottom: 1em;
      }
      #wrapperlicious .permalink {
        display: none;
        left: -0.75em;
        position: absolute;
        padding-right: 0.25em;
        text-decoration: none;
      }
      #wrapperlicious h1:hover .permalink, #wrapperlicious h2:hover .permalink,
      #wrapperlicious h3:hover .permalink, #wrapperlicious h4:hover .permalink,
      #wrapperlicious dt:hover .permalink {
        display: block;
      }
    </style>
    <!-- Global site tag (gtag.js) - Google Analytics -->
    <script async src="https://www.googletagmanager.com/gtag/js?id=G-KVNWBNT5FB"></script>
    <script>
      window.dataLayer = window.dataLayer || [];
      function gtag(){dataLayer.push(arguments);}
      gtag('js', new Date());

      gtag('config', 'G-KVNWBNT5FB');
      gtag('config', 'UA-50555-3');
    </script>
  </head>
  <body>
    <nav class="navbar navbar-expand-md navbar-dark bg-dark justify-content-between">
  <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarNav" aria-controls="navbarNav" aria-expanded="false" aria-label="Toggle navigation">
    <span class="navbar-toggler-icon"></span>
  </button>
  <a class="navbar-brand" href="https://perldoc.perl.org/">Perldoc Browser</a>
  <div class="collapse navbar-collapse" id="navbarNav">
    <ul class="navbar-nav mr-auto">
      <li class="nav-item dropdown text-nowrap">
        <a class="nav-link dropdown-toggle" href="./Test::Harness::Beyond.html#" id="dropdownlink-stable" role="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">5.35.0</a>
        <div class="dropdown-menu" aria-labelledby="dropdownlink-stable">
          <a class="dropdown-item" href="https://perldoc.perl.org/Test::Harness::Beyond">Latest</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.38.2/Test::Harness::Beyond">5.38.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.38.1/Test::Harness::Beyond">5.38.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.38.0/Test::Harness::Beyond">5.38.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.36.3/Test::Harness::Beyond">5.36.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.36.2/Test::Harness::Beyond">5.36.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.36.1/Test::Harness::Beyond">5.36.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.36.0/Test::Harness::Beyond">5.36.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.34.3/Test::Harness::Beyond">5.34.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.34.2/Test::Harness::Beyond">5.34.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.34.1/Test::Harness::Beyond">5.34.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.34.0/Test::Harness::Beyond">5.34.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.32.1/Test::Harness::Beyond">5.32.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.32.0/Test::Harness::Beyond">5.32.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.30.3/Test::Harness::Beyond">5.30.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.30.2/Test::Harness::Beyond">5.30.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.30.1/Test::Harness::Beyond">5.30.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.30.0/Test::Harness::Beyond">5.30.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.28.3/Test::Harness::Beyond">5.28.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.28.2/Test::Harness::Beyond">5.28.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.28.1/Test::Harness::Beyond">5.28.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.28.0/Test::Harness::Beyond">5.28.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.26.3/Test::Harness::Beyond">5.26.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.26.2/Test::Harness::Beyond">5.26.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.26.1/Test::Harness::Beyond">5.26.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.26.0/Test::Harness::Beyond">5.26.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.24.4/Test::Harness::Beyond">5.24.4</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.24.3/Test::Harness::Beyond">5.24.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.24.2/Test::Harness::Beyond">5.24.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.24.1/Test::Harness::Beyond">5.24.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.24.0/Test::Harness::Beyond">5.24.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.22.4/Test::Harness::Beyond">5.22.4</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.22.3/Test::Harness::Beyond">5.22.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.22.2/Test::Harness::Beyond">5.22.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.22.1/Test::Harness::Beyond">5.22.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.22.0/Test::Harness::Beyond">5.22.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.20.3/Test::Harness::Beyond">5.20.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.20.2/Test::Harness::Beyond">5.20.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.20.1/Test::Harness::Beyond">5.20.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.20.0/Test::Harness::Beyond">5.20.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.18.4/Test::Harness::Beyond">5.18.4</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.18.3/Test::Harness::Beyond">5.18.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.18.2/Test::Harness::Beyond">5.18.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.18.1/Test::Harness::Beyond">5.18.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.18.0/Test::Harness::Beyond">5.18.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.16.3/Test::Harness::Beyond">5.16.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.16.2/Test::Harness::Beyond">5.16.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.16.1/Test::Harness::Beyond">5.16.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.16.0/Test::Harness::Beyond">5.16.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.14.4/Test::Harness::Beyond">5.14.4</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.14.3/Test::Harness::Beyond">5.14.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.14.2/Test::Harness::Beyond">5.14.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.14.1/Test::Harness::Beyond">5.14.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.14.0/Test::Harness::Beyond">5.14.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.12.5/Test::Harness::Beyond">5.12.5</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.12.4/Test::Harness::Beyond">5.12.4</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.12.3/Test::Harness::Beyond">5.12.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.12.2/Test::Harness::Beyond">5.12.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.12.1/Test::Harness::Beyond">5.12.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.12.0/Test::Harness::Beyond">5.12.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.10.1/Test::Harness::Beyond">5.10.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.10.0/Test::Harness::Beyond">5.10.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.8.9/Test::Harness::Beyond">5.8.9</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.8.8/Test::Harness::Beyond">5.8.8</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.8.7/Test::Harness::Beyond">5.8.7</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.8.6/Test::Harness::Beyond">5.8.6</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.8.5/Test::Harness::Beyond">5.8.5</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.8.4/Test::Harness::Beyond">5.8.4</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.8.3/Test::Harness::Beyond">5.8.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.8.2/Test::Harness::Beyond">5.8.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.8.1/Test::Harness::Beyond">5.8.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.8.0/Test::Harness::Beyond">5.8.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.6.2/Test::Harness::Beyond">5.6.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.6.1/Test::Harness::Beyond">5.6.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.6.0/Test::Harness::Beyond">5.6.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.005_04/Test::Harness::Beyond">5.005_04</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.005_03/Test::Harness::Beyond">5.005_03</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.005_02/Test::Harness::Beyond">5.005_02</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.005_01/Test::Harness::Beyond">5.005_01</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.005/Test::Harness::Beyond">5.005</a>
        </div>
      </li>
      <li class="nav-item dropdown text-nowrap">
        <a class="nav-link dropdown-toggle" href="./Test::Harness::Beyond.html#" id="dropdownlink-dev" role="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Dev</a>
        <div class="dropdown-menu" aria-labelledby="dropdownlink-dev">
          <a class="dropdown-item" href="https://perldoc.perl.org/blead/Test::Harness::Beyond">blead</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.39.6/Test::Harness::Beyond">5.39.6</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.39.5/Test::Harness::Beyond">5.39.5</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.39.4/Test::Harness::Beyond">5.39.4</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.39.3/Test::Harness::Beyond">5.39.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.39.2/Test::Harness::Beyond">5.39.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.39.1/Test::Harness::Beyond">5.39.1</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.38.0-RC2/Test::Harness::Beyond">5.38.0-RC2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.38.0-RC1/Test::Harness::Beyond">5.38.0-RC1</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.37.11/Test::Harness::Beyond">5.37.11</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.37.10/Test::Harness::Beyond">5.37.10</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.37.9/Test::Harness::Beyond">5.37.9</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.37.8/Test::Harness::Beyond">5.37.8</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.37.7/Test::Harness::Beyond">5.37.7</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.37.6/Test::Harness::Beyond">5.37.6</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.37.5/Test::Harness::Beyond">5.37.5</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.37.4/Test::Harness::Beyond">5.37.4</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.37.3/Test::Harness::Beyond">5.37.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.37.2/Test::Harness::Beyond">5.37.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.37.1/Test::Harness::Beyond">5.37.1</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.37.0/Test::Harness::Beyond">5.37.0</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.35.11/Test::Harness::Beyond">5.35.11</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.35.10/Test::Harness::Beyond">5.35.10</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.35.9/Test::Harness::Beyond">5.35.9</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.35.8/Test::Harness::Beyond">5.35.8</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.35.7/Test::Harness::Beyond">5.35.7</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.35.6/Test::Harness::Beyond">5.35.6</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.35.5/Test::Harness::Beyond">5.35.5</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.35.4/Test::Harness::Beyond">5.35.4</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.35.3/Test::Harness::Beyond">5.35.3</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.35.2/Test::Harness::Beyond">5.35.2</a>
          <a class="dropdown-item" href="https://perldoc.perl.org/5.35.1/Test::Harness::Beyond">5.35.1</a>
          <a class="dropdown-item active" href="./Test::Harness::Beyond.html">5.35.0</a>
        </div>
      </li>
      <li class="nav-item dropdown text-nowrap">
        <a class="nav-link dropdown-toggle" href="./Test::Harness::Beyond.html#" id="dropdownlink-nav" role="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Documentation</a>
        <div class="dropdown-menu" aria-labelledby="dropdownlink-nav">
          <a class="dropdown-item" href="perl.html">Perl</a>
          <a class="dropdown-item" href="perlintro.html">Intro</a>
          <a class="dropdown-item" href="perl.html#Tutorials">Tutorials</a>
          <a class="dropdown-item" href="perlfaq.html">FAQs</a>
          <a class="dropdown-item" href="perl.html#Reference-Manual">Reference</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="perlop.html">Operators</a>
          <a class="dropdown-item" href="functions.html">Functions</a>
          <a class="dropdown-item" href="variables.html">Variables</a>
          <a class="dropdown-item" href="modules.html">Modules</a>
          <a class="dropdown-item" href="perlutil.html">Utilities</a>
          <div class="dropdown-divider"></div>
          <a class="dropdown-item" href="perlcommunity.html">Community</a>
          <a class="dropdown-item" href="perlhist.html">History</a>
        </div>
      </li>
    </ul>
    <ul class="navbar-nav">
      <script>
        function set_expand (expand) {
          var perldocdiv = document.getElementById('perldocdiv');
          var width = window.getComputedStyle(perldocdiv).getPropertyValue('max-width');
          var expanded = (width == '' || width == 'none') ? true : false;
          if (expand === null) {
            expand = !expanded;
          }
          if ((expand && !expanded) || (!expand && expanded)) {
            perldocdiv.style.setProperty('max-width', expand ? 'none' : '960px');
            var button_classlist = document.getElementById('content-expand-button').classList;
            if (expand) {
              button_classlist.add('btn-light');
              button_classlist.remove('btn-outline-light');
            } else {
              button_classlist.add('btn-outline-light');
              button_classlist.remove('btn-light');
            }
          }
          return expand;
        }
        function toggle_expand () {
          var expand = set_expand(null);
          document.cookie = 'perldoc_expand=' + (expand ? 1 : 0) + '; path=/; expires=Tue, 19 Jan 2038 03:14:07 UTC';
        }
        function read_expand () {
          return document.cookie.split(';').some(function (item) { return item.indexOf('perldoc_expand=1') >= 0 });
        }
        if (document.readyState === 'loading') {
          document.addEventListener('DOMContentLoaded', function () {
            if (read_expand()) {
              set_expand(true);
            }
          });
        } else if (read_expand()) {
          set_expand(true);
        }
      </script>
      <button id="content-expand-button" type="button" class="btn btn-outline-light d-none d-lg-inline-block mr-2" onclick="toggle_expand()">Expand</button>
    <ul>
    <form class="form-inline" method="get" action="https://perldoc.perl.org/5.35.0/search">
      <input class="form-control mr-3" type="search" name="q" placeholder="Search" aria-label="Search">
    </form>
  </div>
</nav>

    <div id="wrapperlicious" class="container-fluid">
      <div id="perldocdiv">
        <div id="links">
          <a href="./Test::Harness::Beyond.html">Test::Harness::Beyond</a>
          <div id="more">
            (<a href="./Test::Harness::Beyond.txt">source</a>,
            <a href="https://metacpan.org/pod/Test::Harness::Beyond">CPAN</a>)
          </div>
        </div>
        <div class="leading-notice">
          You are viewing the version of this documentation from Perl 5.35.0.
            This is a development version of Perl.
        </div>
        <h1><a id="toc">CONTENTS</a></h1>
                  <ul>
              <li>
                <a class="text-decoration-none" href="./Test::Harness::Beyond.html#NAME">NAME</a>
              </li>
              <li>
                <a class="text-decoration-none" href="./Test::Harness::Beyond.html#Beyond-make-test">Beyond make test</a>
                            <ul>
              <li>
                <a class="text-decoration-none" href="./Test::Harness::Beyond.html#Saved-State">Saved State</a>
              </li>
              <li>
                <a class="text-decoration-none" href="./Test::Harness::Beyond.html#Parallel-Testing">Parallel Testing</a>
              </li>
              <li>
                <a class="text-decoration-none" href="./Test::Harness::Beyond.html#Non-Perl-Tests">Non-Perl Tests</a>
              </li>
              <li>
                <a class="text-decoration-none" href="./Test::Harness::Beyond.html#Mixing-it-up">Mixing it up</a>
              </li>
              <li>
                <a class="text-decoration-none" href="./Test::Harness::Beyond.html#Rolling-My-Own">Rolling My Own</a>
              </li>
              <li>
                <a class="text-decoration-none" href="./Test::Harness::Beyond.html#Deeper-Customisation">Deeper Customisation</a>
              </li>
              <li>
                <a class="text-decoration-none" href="./Test::Harness::Beyond.html#Callbacks">Callbacks</a>
              </li>
              <li>
                <a class="text-decoration-none" href="./Test::Harness::Beyond.html#Parsing-TAP">Parsing TAP</a>
              </li>
              <li>
                <a class="text-decoration-none" href="./Test::Harness::Beyond.html#Getting-Support">Getting Support</a>
              </li>
          </ul>

              </li>
          </ul>

      <h1 id="NAME"><a class="permalink" href="./Test::Harness::Beyond.html#NAME">#</a>NAME</h1>

<p>Test::Harness::Beyond - Beyond make test</p>

<h1 id="Beyond-make-test"><a class="permalink" href="./Test::Harness::Beyond.html#Beyond-make-test">#</a><a id="Beyond"></a>Beyond make test</h1>

<p>Test::Harness is responsible for running test scripts, analysing their output and reporting success or failure. When I type <i>make test</i> (or <i>./Build test</i>) for a module, Test::Harness is usually used to run the tests (not all modules use Test::Harness but the majority do).</p>

<p>To start exploring some of the features of Test::Harness I need to switch from <i>make test</i> to the <i>prove</i> command (which ships with Test::Harness). For the following examples I&#39;ll also need a recent version of Test::Harness installed; 3.14 is current as I write.</p>

<p>For the examples I&#39;m going to assume that we&#39;re working with a &#39;normal&#39; Perl module distribution. Specifically I&#39;ll assume that typing <i>make</i> or <i>./Build</i> causes the built, ready-to-install module code to be available below ./blib/lib and ./blib/arch and that there&#39;s a directory called &#39;t&#39; that contains our tests. Test::Harness isn&#39;t hardwired to that configuration but it saves me from explaining which files live where for each example.</p>

<p>Back to <i>prove</i>; like <i>make test</i> it runs a test suite - but it provides far more control over which tests are executed, in what order and how their results are reported. Typically <i>make test</i> runs all the test scripts below the &#39;t&#39; directory. To do the same thing with prove I type:</p>

<pre><code class="plaintext">prove -rb t</code></pre>

<p>The switches here are -r to recurse into any directories below &#39;t&#39; and -b which adds ./blib/lib and ./blib/arch to Perl&#39;s include path so that the tests can find the code they will be testing. If I&#39;m testing a module of which an earlier version is already installed I need to be careful about the include path to make sure I&#39;m not running my tests against the installed version rather than the new one that I&#39;m working on.</p>

<p>Unlike <i>make test</i>, typing <i>prove</i> doesn&#39;t automatically rebuild my module. If I forget to make before prove I will be testing against older versions of those files - which inevitably leads to confusion. I either get into the habit of typing</p>

<pre><code class="plaintext">make &amp;&amp; prove -rb t</code></pre>

<p>or - if I have no XS code that needs to be built I use the modules below <i>lib</i> instead</p>

<pre><code class="plaintext">prove -Ilib -r t</code></pre>

<p>So far I&#39;ve shown you nothing that <i>make test</i> doesn&#39;t do. Let&#39;s fix that.</p>

<h2 id="Saved-State"><a class="permalink" href="./Test::Harness::Beyond.html#Saved-State">#</a><a id="Saved"></a>Saved State</h2>

<p>If I have failing tests in a test suite that consists of more than a handful of scripts and takes more than a few seconds to run it rapidly becomes tedious to run the whole test suite repeatedly as I track down the problems.</p>

<p>I can tell prove just to run the tests that are failing like this:</p>

<pre><code class="plaintext">prove -b t/this_fails.t t/so_does_this.t</code></pre>

<p>That speeds things up but I have to make a note of which tests are failing and make sure that I run those tests. Instead I can use prove&#39;s --state switch and have it keep track of failing tests for me. First I do a complete run of the test suite and tell prove to save the results:</p>

<pre><code class="plaintext">prove -rb --state=save t</code></pre>

<p>That stores a machine readable summary of the test run in a file called &#39;.prove&#39; in the current directory. If I have failures I can then run just the failing scripts like this:</p>

<pre><code class="plaintext">prove -b --state=failed</code></pre>

<p>I can also tell prove to save the results again so that it updates its idea of which tests failed:</p>

<pre><code class="plaintext">prove -b --state=failed,save</code></pre>

<p>As soon as one of my failing tests passes it will be removed from the list of failed tests. Eventually I fix them all and prove can find no failing tests to run:</p>

<pre><code class="plaintext">Files=0, Tests=0, 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU)
Result: NOTESTS</code></pre>

<p>As I work on a particular part of my module it&#39;s most likely that the tests that cover that code will fail. I&#39;d like to run the whole test suite but have it prioritize these &#39;hot&#39; tests. I can tell prove to do this:</p>

<pre><code class="plaintext">prove -rb --state=hot,save t</code></pre>

<p>All the tests will run but those that failed most recently will be run first. If no tests have failed since I started saving state all tests will run in their normal order. This combines full test coverage with early notification of failures.</p>

<p>The --state switch supports a number of options; for example to run failed tests first followed by all remaining tests ordered by the timestamps of the test scripts - and save the results - I can use</p>

<pre><code class="plaintext">prove -rb --state=failed,new,save t</code></pre>

<p>See the prove documentation (type prove --man) for the full list of state options.</p>

<p>When I tell prove to save state it writes a file called &#39;.prove&#39; (&#39;_prove&#39; on Windows) in the current directory. It&#39;s a YAML document so it&#39;s quite easy to write tools of your own that work on the saved test state - but the format isn&#39;t officially documented so it might change without (much) warning in the future.</p>

<h2 id="Parallel-Testing"><a class="permalink" href="./Test::Harness::Beyond.html#Parallel-Testing">#</a><a id="Parallel"></a>Parallel Testing</h2>

<p>If my tests take too long to run I may be able to speed them up by running multiple test scripts in parallel. This is particularly effective if the tests are I/O bound or if I have multiple CPU cores. I tell prove to run my tests in parallel like this:</p>

<pre><code class="plaintext">prove -rb -j 9 t</code></pre>

<p>The -j switch enables parallel testing; the number that follows it is the maximum number of tests to run in parallel. Sometimes tests that pass when run sequentially will fail when run in parallel. For example if two different test scripts use the same temporary file or attempt to listen on the same socket I&#39;ll have problems running them in parallel. If I see unexpected failures I need to check my tests to work out which of them are trampling on the same resource and rename temporary files or add locks as appropriate.</p>

<p>To get the most performance benefit I want to have the test scripts that take the longest to run start first - otherwise I&#39;ll be waiting for the one test that takes nearly a minute to complete after all the others are done. I can use the --state switch to run the tests in slowest to fastest order:</p>

<pre><code class="plaintext">prove -rb -j 9 --state=slow,save t</code></pre>

<h2 id="Non-Perl-Tests"><a class="permalink" href="./Test::Harness::Beyond.html#Non-Perl-Tests">#</a><a id="Non"></a>Non-Perl Tests</h2>

<p>The Test Anything Protocol (http://testanything.org/) isn&#39;t just for Perl. Just about any language can be used to write tests that output TAP. There are TAP based testing libraries for C, C++, PHP, Python and many others. If I can&#39;t find a TAP library for my language of choice it&#39;s easy to generate valid TAP. It looks like this:</p>

<pre><code class="plaintext">1..3 
ok 1 - init OK 
ok 2 - opened file 
not ok 3 - appended to file</code></pre>

<p>The first line is the plan - it specifies the number of tests I&#39;m going to run so that it&#39;s easy to check that the test script didn&#39;t exit before running all the expected tests. The following lines are the test results - &#39;ok&#39; for pass, &#39;not ok&#39; for fail. Each test has a number and, optionally, a description. And that&#39;s it. Any language that can produce output like that on STDOUT can be used to write tests.</p>

<p>Recently I&#39;ve been rekindling a two-decades-old interest in Forth. Evidently I have a masochistic streak that even Perl can&#39;t satisfy. I want to write tests in Forth and run them using prove (you can find my gforth TAP experiments at https://svn.hexten.net/andy/Forth/Testing/). I can use the --exec switch to tell prove to run the tests using gforth like this:</p>

<pre><code class="plaintext">prove -r --exec gforth t</code></pre>

<p>Alternately, if the language used to write my tests allows a shebang line I can use that to specify the interpreter. Here&#39;s a test written in PHP:</p>

<pre><code>#!/usr/bin/php 
&lt;?php
  print &quot;1..2\n&quot;; 
  print &quot;ok 1\n&quot;; 
  print &quot;not ok 2\n&quot;;
?&gt;</code></pre>

<p>If I save that as t/phptest.t the shebang line will ensure that it runs correctly along with all my other tests.</p>

<h2 id="Mixing-it-up"><a class="permalink" href="./Test::Harness::Beyond.html#Mixing-it-up">#</a><a id="Mixing"></a>Mixing it up</h2>

<p>Subtle interdependencies between test programs can mask problems - for example an earlier test may neglect to remove a temporary file that affects the behaviour of a later test. To find this kind of problem I use the --shuffle and --reverse options to run my tests in random or reversed order.</p>

<h2 id="Rolling-My-Own"><a class="permalink" href="./Test::Harness::Beyond.html#Rolling-My-Own">#</a><a id="Rolling"></a>Rolling My Own</h2>

<p>If I need a feature that prove doesn&#39;t provide I can easily write my own.</p>

<p>Typically you&#39;ll want to change how TAP gets <i>input</i> into and <i>output</i> from the parser. <a href="./App::Prove.html">App::Prove</a> supports arbitrary plugins, and <a href="./TAP::Harness.html">TAP::Harness</a> supports custom <i>formatters</i> and <i>source handlers</i> that you can load using either <a href="prove.html">prove</a> or <a href="./Module::Build.html">Module::Build</a>; there are many examples to base mine on. For more details see <a href="./App::Prove.html">App::Prove</a>, <a href="./TAP::Parser::SourceHandler.html">TAP::Parser::SourceHandler</a>, and <a href="./TAP::Formatter::Base.html">TAP::Formatter::Base</a>.</p>

<p>If writing a plugin is not enough, you can write your own test harness; one of the motives for the 3.00 rewrite of Test::Harness was to make it easier to subclass and extend.</p>

<p>The Test::Harness module is a compatibility wrapper around TAP::Harness. For new applications I should use TAP::Harness directly. As we&#39;ll see, prove uses TAP::Harness.</p>

<p>When I run prove it processes its arguments, figures out which test scripts to run and then passes control to TAP::Harness to run the tests, parse, analyse and present the results. By subclassing TAP::Harness I can customise many aspects of the test run.</p>

<p>I want to log my test results in a database so I can track them over time. To do this I override the summary method in TAP::Harness. I start with a simple prototype that dumps the results as a YAML document:</p>

<pre><code>package My::TAP::Harness;

use base &#39;TAP::Harness&#39;;
use YAML;

sub summary {
  my ( $self, $aggregate ) = @_; 
  print Dump( $aggregate );
  $self-&gt;SUPER::summary( $aggregate );
}

1;</code></pre>

<p>I need to tell prove to use my My::TAP::Harness. If My::TAP::Harness is on Perl&#39;s @INC include path I can</p>

<pre><code class="plaintext">prove --harness=My::TAP::Harness -rb t</code></pre>

<p>If I don&#39;t have My::TAP::Harness installed on @INC I need to provide the correct path to perl when I run prove:</p>

<pre><code class="plaintext">perl -Ilib `which prove` --harness=My::TAP::Harness -rb t</code></pre>

<p>I can incorporate these options into my own version of prove. It&#39;s pretty simple. Most of the work of prove is handled by App::Prove. The important code in prove is just:</p>

<pre><code>use App::Prove;

my $app = App::Prove-&gt;new; 
$app-&gt;process_args(@ARGV); 
exit( $app-&gt;run ? 0 : 1 );</code></pre>

<p>If I write a subclass of App::Prove I can customise any aspect of the test runner while inheriting all of prove&#39;s behaviour. Here&#39;s myprove:</p>

<pre><code>#!/usr/bin/env perl use lib qw( lib );      # Add ./lib to @INC
use App::Prove;

my $app = App::Prove-&gt;new;

# Use custom TAP::Harness subclass
$app-&gt;harness( &#39;My::TAP::Harness&#39; );

$app-&gt;process_args( @ARGV ); exit( $app-&gt;run ? 0 : 1 );</code></pre>

<p>Now I can run my tests like this</p>

<pre><code class="plaintext">./myprove -rb t</code></pre>

<h2 id="Deeper-Customisation"><a class="permalink" href="./Test::Harness::Beyond.html#Deeper-Customisation">#</a><a id="Deeper"></a>Deeper Customisation</h2>

<p>Now that I know how to subclass and replace TAP::Harness I can replace any other part of the harness. To do that I need to know which classes are responsible for which functionality. Here&#39;s a brief guided tour; the default class for each component is shown in parentheses. Normally any replacements I write will be subclasses of these default classes.</p>

<p>When I run my tests TAP::Harness creates a scheduler (TAP::Parser::Scheduler) to work out the running order for the tests, an aggregator (TAP::Parser::Aggregator) to collect and analyse the test results and a formatter (TAP::Formatter::Console) to display those results.</p>

<p>If I&#39;m running my tests in parallel there may also be a multiplexer (TAP::Parser::Multiplexer) - the component that allows multiple tests to run simultaneously.</p>

<p>Once it has created those helpers TAP::Harness starts running the tests. For each test it creates a new parser (TAP::Parser) which is responsible for running the test script and parsing its output.</p>

<p>To replace any of these components I call one of these harness methods with the name of the replacement class:</p>

<pre><code class="plaintext">aggregator_class 
formatter_class 
multiplexer_class 
parser_class
scheduler_class</code></pre>

<p>For example, to replace the aggregator I would</p>

<pre><code>$harness-&gt;aggregator_class( &#39;My::Aggregator&#39; );</code></pre>

<p>Alternately I can supply the names of my substitute classes to the TAP::Harness constructor:</p>

<pre><code>my $harness = TAP::Harness-&gt;new(
  { aggregator_class =&gt; &#39;My::Aggregator&#39; }
);</code></pre>

<p>If I need to reach even deeper into the internals of the harness I can replace the classes that TAP::Parser uses to execute test scripts and tokenise their output. Before running a test script TAP::Parser creates a grammar (TAP::Parser::Grammar) to decode the raw TAP into tokens, a result factory (TAP::Parser::ResultFactory) to turn the decoded TAP results into objects and, depending on whether it&#39;s running a test script or reading TAP from a file, scalar or array a source or an iterator (TAP::Parser::IteratorFactory).</p>

<p>Each of these objects may be replaced by calling one of these parser methods:</p>

<pre><code class="plaintext">source_class
perl_source_class 
grammar_class 
iterator_factory_class
result_factory_class</code></pre>

<h2 id="Callbacks"><a class="permalink" href="./Test::Harness::Beyond.html#Callbacks">#</a>Callbacks</h2>

<p>As an alternative to subclassing the components I need to change I can attach callbacks to the default classes. TAP::Harness exposes these callbacks:</p>

<pre><code class="plaintext">parser_args      Tweak the parameters used to create the parser 
made_parser      Just made a new parser 
before_runtests  About to run tests 
after_runtests   Have run all tests 
after_test       Have run an individual test script</code></pre>

<p>TAP::Parser also supports callbacks; bailout, comment, plan, test, unknown, version and yaml are called for the corresponding TAP result types, ALL is called for all results, ELSE is called for all results for which a named callback is not installed and EOF is called once at the end of each TAP stream.</p>

<p>To install a callback I pass the name of the callback and a subroutine reference to TAP::Harness or TAP::Parser&#39;s callback method:</p>

<pre><code>$harness-&gt;callback( after_test =&gt; sub {
  my ( $script, $desc, $parser ) = @_;
} );</code></pre>

<p>I can also pass callbacks to the constructor:</p>

<pre><code>  my $harness = TAP::Harness-&gt;new({
    callbacks =&gt; {
	    after_test =&gt; sub {
        my ( $script, $desc, $parser ) = @_; 
        # Do something interesting here
	    }
    }
  });</code></pre>

<p>When it comes to altering the behaviour of the test harness there&#39;s more than one way to do it. Which way is best depends on my requirements. In general if I only want to observe test execution without changing the harness&#39; behaviour (for example to log test results to a database) I choose callbacks. If I want to make the harness behave differently subclassing gives me more control.</p>

<h2 id="Parsing-TAP"><a class="permalink" href="./Test::Harness::Beyond.html#Parsing-TAP">#</a><a id="Parsing"></a>Parsing TAP</h2>

<p>Perhaps I don&#39;t need a complete test harness. If I already have a TAP test log that I need to parse all I need is TAP::Parser and the various classes it depends upon. Here&#39;s the code I need to run a test and parse its TAP output</p>

<pre><code>use TAP::Parser;

my $parser = TAP::Parser-&gt;new( { source =&gt; &#39;t/simple.t&#39; } );
while ( my $result = $parser-&gt;next ) {
  print $result-&gt;as_string, &quot;\n&quot;;
}</code></pre>

<p>Alternately I can pass an open filehandle as source and have the parser read from that rather than attempting to run a test script:</p>

<pre><code>open my $tap, &#39;&lt;&#39;, &#39;tests.tap&#39; 
  or die &quot;Can&#39;t read TAP transcript ($!)\n&quot;; 
my $parser = TAP::Parser-&gt;new( { source =&gt; $tap } );
while ( my $result = $parser-&gt;next ) {
  print $result-&gt;as_string, &quot;\n&quot;;
}</code></pre>

<p>This approach is useful if I need to convert my TAP based test results into some other representation. See TAP::Convert::TET (http://search.cpan.org/dist/TAP-Convert-TET/) for an example of this approach.</p>

<h2 id="Getting-Support"><a class="permalink" href="./Test::Harness::Beyond.html#Getting-Support">#</a><a id="Getting"></a>Getting Support</h2>

<p>The Test::Harness developers hang out on the tapx-dev mailing list[1]. For discussion of general, language independent TAP issues there&#39;s the tap-l[2] list. Finally there&#39;s a wiki dedicated to the Test Anything Protocol[3]. Contributions to the wiki, patches and suggestions are all welcome.</p>

<p>[1] <a href="http://www.hexten.net/mailman/listinfo/tapx-dev">http://www.hexten.net/mailman/listinfo/tapx-dev</a> [2] <a href="http://testanything.org/mailman/listinfo/tap-l">http://testanything.org/mailman/listinfo/tap-l</a> [3] <a href="http://testanything.org/">http://testanything.org/</a></p>


      </div>
      <div id="footer">
        <p>Perldoc Browser is maintained by Dan Book (<a href="https://metacpan.org/author/DBOOK">DBOOK</a>). Please contact him via the <a href="https://github.com/Grinnz/perldoc-browser/issues">GitHub issue tracker</a> or <a href="mailto:dbook@cpan.org">email</a> regarding any issues with the site itself, search, or rendering of documentation.</p>

<p>The Perl documentation is maintained by the Perl 5 Porters in the development of Perl. Please contact them via the <a href="https://github.com/Perl/perl5/issues">Perl issue tracker</a>, the <a href="https://lists.perl.org/list/perl5-porters.html">mailing list</a>, or <a href="https://kiwiirc.com/client/irc.perl.org/p5p">IRC</a> to report any issues with the contents or format of the documentation.</p>


      </div>
    </div>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.5.1/jquery.slim.min.js" integrity="sha512-/DXTXr6nQodMUiq+IUJYCt2PPOUjrHJ9wFrqpJ3XkgPNOZVfMok7cRw6CSxyCQxXn6ozlESsSh1/sMCTF1rL/g==" crossorigin="anonymous"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.16.1/umd/popper.min.js" integrity="sha512-ubuT8Z88WxezgSqf3RLuNi5lmjstiJcyezx34yIU2gAHonIi27Na7atqzUZCOoY4CExaoFumzOsFQ2Ch+I/HCw==" crossorigin="anonymous"></script>
    <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/js/bootstrap.min.js" integrity="sha384-B4gt1jrGC7Jh4AgTPSdUtOBvfO8shuf57BaghqFfPlYxofvL8/KUEfYiJOMMV+rV" crossorigin="anonymous"></script>
    <script src="../highlight.pack.js"></script>
    <script>hljs.highlightAll();</script>
  </body>
</html>
